Skip to content

Fix fragment-pop stale stroke: realign raw buffer to truncated word#73

Open
SHAWNERZZ wants to merge 25 commits into
AsafMah:mainfrom
SHAWNERZZ:fix/fragment-pop-stale-stroke
Open

Fix fragment-pop stale stroke: realign raw buffer to truncated word#73
SHAWNERZZ wants to merge 25 commits into
AsafMah:mainfrom
SHAWNERZZ:fix/fragment-pop-stale-stroke

Conversation

@SHAWNERZZ

Copy link
Copy Markdown

Problem

Two-thumb re-recognition: swipe th + swipe ing builds "thing", but backspacing the ing fragment and re-swiping ing produced "thinking". The fragment-pop correctly truncated the text, but the raw stroke buffer (mInputPointers) still held the longer pre-pop geometry. The next swipe-extend snapshotted that stale buffer as its merged-trail base, so the recognizer saw ~26 stale points + the new gesture and rebuilt an ever-longer word.

This is Phase 1 of docs/COMPOSING_WORD_SOURCE_OF_TRUTH.md (editor text as source of truth).

Fix

  • tryFragmentBackspace realigns the raw stroke buffer to the truncated word's key centers via seedInputPointersFromKeyCenters, and lowercases the word first — Keyboard#getCoordinates is an exact code-point match and layouts store lowercase, so an uppercase Th would resolve to NOT_A_COORDINATE and silently drop out of the seed.
  • seedInputPointersFromKeyCenters skips NOT_A_COORDINATE points so an unresolvable key can't warp the stroke toward (-1,-1).
  • Added getExtendBatchInputBaseSize() and diagnostic log lines (gated on mGestureDebugDrawPoints) so the buffer state is observable on-device.

Tests

  • New end-to-end testFragmentPopThenReswipeUsesSeededBaseNotStaleTrail: replays the failing sequence and asserts the re-armed base is the 2-point seed (27 merged points, not ~51).
  • New testSeedInputPointersSkipsUnresolvableKeys.
  • :app:testOfflineDebugUnitTest --tests WordComposerTest passes.

Validation

On-device: th + ing + backspace + re-swipe ing now yields "thing". Confirmed by the user.

🤖 Generated with Claude Code

SHAWNERZZ and others added 25 commits June 6, 2026 14:53
Captures the opt-in learned key-geometry feature: one per-user model behind both
taps (KeyDetector) and gestures (ProximityInfo sweet spots, Java-only / no native
rebuild), the context-prior + learned-geometry layers, hard caps to avoid wrong
literals, privacy (content-free, incognito-gated), leantype.db persistence riding
the existing backup, a stats page, and a phased build order.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…store)

Phase 1 of #1 (no behavior yet; nothing reads the model). Adds the opt-in prefs
PREF_ADAPTIVE_KEY_GEOMETRY (+ strength) via the 5-file pattern, and a content-free
per-(key, layout, orientation) touch-model table in leantype.db with a cached DAO
(EMA running mean/variance + count, restore, clear).

DB VERSION 2->3 with an additive onUpgrade, and copyFromDb extended to carry the
model across the existing settings backup/restore (guarded so older backups without
the table restore fine). Compiles: :app:compileStandardDebugJavaWithJavac.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Phase 2 policy layer for #1: capped, confidence- and strength-scaled landing-offset
math (pure functions). The cap (MAX_SHIFT_FRACTION of key size) is the safety bound
so a learned bias can never flip a clear press to a neighbor; confidence ramps the
bias in with sample count; strength scales it (0 = off). 7 unit tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…on + settings

Phase 2 of #1 (testable end-to-end). Closes the loop:
- PointerTracker records each letter tap's landing offset into the touch model
  (opt-in + incognito gated; async DB write so typing stays fast).
- ProximityInfo shifts per-key sweet spots by the capped learned offset, so gesture
  recognition AND tap-correction follow where the user actually types. Generated even
  when a layout has no static touch-position-correction data. Element id passed from
  Keyboard; computed in Java, crosses the existing JNI - no native change.
- Gesture typing -> Advanced: opt-in toggle + strength slider (reload keyboard on change).
- @JvmStatic on the Kotlin DAO factory / manager so the Java hot paths can call them.

Deferred: literal-tap KeyDetector tie-break (marginal/risky), the completion-derived
context prior, and the stats page. Builds: :app:assembleStandardDebug.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…page

#1, follow-ups to the adaptive-geometry feature:
- Gestures now teach the model (not just consume it) via their clean endpoints:
  finger-down -> word's first letter, finger-up -> last letter. Interior keys skipped
  (corner-cutting); fresh single strokes only. InputLogic.maybeRecordGestureEndpoints.
- New stats screen (AdaptiveTypingStatsScreen) reachable from Gesture typing -> Advanced
  when the feature is on: per-key average offset, spread (consistency), sample count,
  plus a Reset button. Wired via SettingsDestination + a clickable Preference row.

Build + SettingsContainerTest green; :app:assembleStandardDebug builds.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ure lifecycle

WordComposer.mExtendBatchInputBase (the multi-part merged-trail base) was only
cleared by a normally-completing gesture, so an abnormal end (cancel / empty-top
recognition) left it armed, and no deletion path cleared it. A later fresh swipe then
merged with the ghost trail - most visibly at the start of a text box.

Clear it at the same word-end sites where mLiveStroke is already dropped
(handleBackspaceEvent, resetComposingState, commitChosenWord, desync) plus the two
gesture-lifecycle origins (onStartBatchInput top, onCancelBatchInput). Does not touch
the hot WordComposer.reset() path.

Adds 8 tests: base cleared after each backspace mode (character/fragment/whole-word),
the delete slider, fresh onStartBatchInput, and onCancelBatchInput; plus 2 guards
pinning the (currently dead) static-seed interlock.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… page

#1. The raw px-delta list was hard to interpret, so show the data spatially:
- Store each key's size (KEY_WIDTH/KEY_HEIGHT) with its offset so the offset can be
  expressed as a fraction of the key (DB VERSION 3->4; additive ALTER on upgrade;
  copyFromDb reads by column name so older backups restore fine).
- Stats page now renders a mock QWERTY where each key shows a dot at where you tend to
  land (offset as a fraction of the key) plus a faint spread ring; confident keys in the
  accent color, still-learning keys faded. The numeric list stays below for exact values.
- Recorders (tap + gesture-endpoint) now pass the key's hitbox size.

Build + extendBase/manager/settings tests green; :app:assembleStandardDebug builds.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Completes adaptive typing on the tap side:
- AdaptiveKeyContext builds a "likely next key" prior from the top-5 suggestions,
  weighted equally (averaged, not score-skewed): the next char of the in-progress
  word's completions, or the first char of next-word predictions for a fresh word.
  Rebuilt between keystrokes in InputLogic.setSuggestedWords (off the tap hot path),
  read lock-free per tap.
- KeyDetector.detectHitKey now biases the tapped key by the learned per-key landing
  offset (shifts the effective center) AND the context prior (enlarges likely keys).
  Both capped; the prior cap (~18% of key) is deliberately below the learned cap
  (~25%) so it nudges rather than dominates. Only near-boundary taps can flip; clear
  presses are untouched.
- Suppressed during gestures/swipes (PointerTracker.isInGestureOrKeySwipe); gestures
  are not context-biased since suggestions don't change mid-swipe.

Applies to both current words being built and the first key of a new word (predictions).
Build + extendBase/manager/settings tests green; :app:assembleStandardDebug builds.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Split the next-key context prior into its own opt-in setting
(adaptive_context_prior, default off) alongside the existing learned
key-geometry toggle, and group both under a single "Adaptive typing"
section in the Gesture typing settings.

- Settings/Defaults/SettingsValues: new mAdaptiveContextPrior flag.
- KeyDetector: independently gate learned-offset bias vs prior boost;
  either alone can bias a near-boundary tap, both share the strength
  slider. adjustedDistance now takes usePrior and gates the boost.
- InputLogic.setSuggestedWords: build the prior only when the prior
  toggle is on; clear AdaptiveKeyContext otherwise.
- GestureTypingScreen: new "Adaptive typing" category holding both
  toggles; strength slider shown if either is on; stats shown if
  learning is on.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A debug toggle ("Show adaptive targets on keyboard") that visualizes the
adaptive model directly on the live keyboard, so the feature is visible as
you type:

- Learned geometry: each letter key shows a faint geometric-center ring, an
  arrow to its learned landing target, and a dot at that target.
- Context prior: keys the suggestions predict get a translucent halo whose
  radius grows with the prior weight; halos morph between keystrokes.

Implemented as AdaptiveTargetsDrawingPreview (an AbstractDrawingPreview, same
mechanism as the gesture-debug overlay), drawn on the DrawingPreviewPlacerView
above the keys. It is purely visual, reads the same live model / prior /
settings the engine uses, and is gated on its pref each frame (zero cost when
off). The halo radius is exaggerated vs the engine's sub-key boost for
legibility; the visible keys are deliberately not reflowed.

- New pref PREF_ADAPTIVE_DEBUG_OVERLAY (5-file), shown under "Adaptive typing"
  when either adaptive toggle is on.
- AdaptiveKeyContext gains a change listener fired on update()/clear(); the
  overlay repaints on it. MainKeyboardView registers the listener and feeds
  the overlay the keyboard + padding so markers align with rendered keys.
- docs/ADAPTIVE_TYPING.md: document the overlay, the independent context-prior
  toggle, the grouped settings, the built heatmap, and fix the DB version.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
On-device tracing showed the next-key bias is applied correctly in
practice (each tap is biased by the prediction for the prefix typed so
far), but two issues made it look/behave wrong:

1. Overlay trailed by ~one key. The prior is rebuilt only when the
   suggestion strip refreshes, which is debounced ~100 ms; the trace
   showed a consistent ~113 ms gap between a keystroke and its
   setSuggested, so for most of the inter-key interval the overlay still
   showed the previous prediction. When the context prior is enabled,
   shorten that debounce (PROMPT_PRIOR_UPDATE_DELAY_MS = 30 ms in
   LatinIME.UIHandler) so the prediction lands before the next tap.

2. Bias silently skipped capital letters. The prior stores lowercase
   next-characters, but a shifted keyboard reports uppercase key codes,
   so AdaptiveKeyContext.weight() missed them (priorOnGeo=0.0 on the
   sentence-initial capital). weight() now folds the queried code to
   lowercase, so the bias and the overlay halos work on the shifted
   layout too.

Also adds debug-overlay-gated tracing (AdaptivePrior tag) in
setSuggestedWords and KeyDetector, plus AdaptiveKeyContext.debugString(),
to make this diagnosable on-device, and documents both fixes in
docs/ADAPTIVE_TYPING.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The suggestion-update debounce (~100 ms) exists because the suggestion
compute blocks the UI thread (performUpdateSuggestionStripSync waits on
holder.get with a 200 ms timeout); the debounce coalesces that expensive
compute so fast typing isn't hit by one block per keystroke.

The functional context-prior bias already works at the full debounce (at
normal speed the prior is ready before the next tap). Only the debug
overlay visibly trails. So instead of shortening the debounce whenever the
prior is enabled, shorten it only while the debug overlay is also on — a
temporary diagnostic — and raise the value from 30 ms to 50 ms. Ordinary
typing (overlay off) keeps the full debounce and is unaffected.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 50 ms (and even 30 ms) overlay debounce was easy to out-type. A truly
non-blocking async refresh isn't safe here: the suggestion compute reads
the non-thread-safe WordComposer on a background thread, and the existing
design keeps that safe by blocking the UI during the compute (the only
existing async compute runs during gestures, when there's no concurrent
typing). Doing it async during typing would risk a torn read / crash and
would need a composer snapshot — too much for a debug visualization.

Instead, drop the suggestion debounce to 0 while the debug overlay + the
context prior are both on. The overlay already repaints the instant the
prior updates (the AdaptiveKeyContext change listener fires inside
setSuggestedWords), so computing immediately is the safe equivalent of
"update as soon as the suggestion is made". The only remaining latency is
the compute itself (~5-10 ms release, ~20 ms debug build). Ordinary typing
(overlay off) keeps the full, smooth debounce.

Reverts the earlier PROMPT_PRIOR_UPDATE_DELAY_MS approach.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Casing bugs specific to two-thumb live-converge / multi-part re-recognition,
where every extending tap or swipe REPLACES the whole composing word with the
recognizer's fresh (lowercase) output:

- Dropped auto-cap (#5): "Hello" -> "hellow" on the first extension.
- Stuck all-caps (#4): a short ambiguous swipe whose top pick is an all-caps
  acronym ("CSA") set WordComposer.isAllUpperCase, which forced every later
  suggestion upper, so the word stuck in caps ("CSA"->"CAN"->"CAME").
- Swipe-extension downcasing: "Was"+swipe -> "wait" (a second swipe re-entered
  onStartBatchInput and re-captured the now-cleared shift state).

Approach: separate a word's casing INTENT from the recognizer's letters.

- WordComposer.mCapitalizedMode is the persistent per-word intent: seeded at
  word start from auto-cap + shift, it survives the setBatchInputWord rebuild
  and is cleared only at commitWord. Exposed via getCapitalizedMode().
- New InputLogic.applyComposingCase(lemma, capsMode, locale) treats the
  recognizer output as a casing-NEUTRAL lemma (lowercased first) and re-applies
  the intent. Lowercasing first dissolves #4 at the source: the composing word
  is never all-caps, so isAllUpperCase never arms.
- onStartBatchInput captures the intent only for a FRESH word (gated on
  !extendComposingWord), so an extending gesture preserves the first fragment's
  intent instead of re-capturing the auto-cleared shift state (#5 / "Wait").
- Fresh-word gesture capitalization is unchanged (still mShiftModeAtGestureStart,
  captured before any state mutates), so plain glide typing is byte-identical
  and a standalone acronym swipe ("CSA") still stays as-is.

Tests: applyComposingCase covered directly (pure, native-free) plus intent
lifecycle tests in InputLogicTest. :app:testOfflineDebugUnitTest green except
the 3 documented pre-existing failures.

Fixes #4, #5.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fix live-converge word casing: stuck all-caps (#4) + dropped auto-cap (#5)
feat(typing): adaptive learned key geometry (taps + gestures)
After a fragment-pop backspace (swipe "th" + swipe "ing", then backspace
to drop "ing"), the raw stroke buffer still held the longer pre-pop
geometry. A following swipe-extend snapshotted that stale buffer as its
merged-trail base and rebuilt an ever-longer word ("thinking" instead of
"thing"). Phase 1 of COMPOSING_WORD_SOURCE_OF_TRUTH.md.

- tryFragmentBackspace now lowercases the truncated word before key-center
  lookup (Keyboard#getCoordinates is an exact code-point match and layouts
  store lowercase, so "Th" would resolve to NOT_A_COORDINATE and drop out).
- seedInputPointersFromKeyCenters skips NOT_A_COORDINATE points so an
  unresolvable key cannot warp the stroke toward (-1,-1).
- Added getExtendBatchInputBaseSize() plus diagnostic log lines at the
  extend-arm site and in the batch-candidates dump (gated on
  mGestureDebugDrawPoints) to make the buffer state observable on-device.
- New tests: end-to-end fragment-pop -> reswipe simulation asserting the
  re-armed base is the 2-point seed (27 merged pts, not ~51), and the
  NOT_A_COORDINATE-skip case.

Validated on-device: th + ing + backspace + reswipe ing now yields "thing".

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant